Inspiration

  • Kentucky’s accountability system re-design decision to forgo growth and utilize improvement instead.
  • Countless discussions at TAC meetings where accountability system terminology is used interchangeably and incorrectly (most notably by experts in our field).
  • Numerous presentations at TAC meetings discussing poorly designed quasi-experimental studies using results from state summative assessments.
  • Characterizations of “learning loss” and “recovery” during and after the COVID-19 pandemic.
  • Numerous staff meeting discussions about accountability systems and their design.
  • 2023 Brian Gong Colloquium on accountability system design.

Coherence

  • The term coherence is not frequently used in discussions of accountability systems and when it is, it is often not clearly defined.
  • What is coherence with respect to accountability systems?
  • What is incoherent about current accountability systems?
  • Why is coherence important?
  • How can coherence be instantiated in accountability systems?

Gill’s Framework

  • Brian Gill’s (2022) paper, What should the future of educational accountability look like?, introduces a useful framework for indicators used in accountability systems.
  • Three Core Components of Gill’s Framework
    • Processes: The activities, instructional practices, and organizational elements that schools implement to support student learning.
    • Impacts: The measurable effects of processes on student learning and development, often captured through growth metrics or value-added models.
    • Outcomes: The long-term results of education, such as achievement/proficiency levels, graduation rates, and other indicators of student success.
  • Gill argues that for indicators to drive meaningful improvement, they must be:
    • Valid: Accurately measure what they claim to measure without systematic bias.
    • Reliable: Stable and consistent over time to avoid misdiagnosis or loss of credibility.
    • Robust: Resistant to unintended consequences and manipulation while capturing critical dimensions of educational quality.

Achievement & Growth/Carlson’s Table

References

Gill, Brian. 2022. “What Should the Future of Educational Accountability Look Like?” Journal of Policy Analysis and Management 41 (4): 1232–39. https://doi.org/10.1002/pam.22428.